Information recognition system and control system using same

ABSTRACT

An information recognition circuit comprises a plurality of recognition processing units each composed of a neural network. Teacher signals and information signals to be processed are supplied to a plurality of the units, individually so as to obtain output signals by executing individual learning. Thereafter, the plural units are connected to each other so as to construct a large scale information recognition system. Further, in the man-machine interface system, a plurality of operating instruction data are prepared. An operator&#39;s face is sensed by a TV camera to extract the factors related to the operator&#39;s facial expression. The neural network analogizes operator&#39;s feeling on the basis of the extracted factors. In accordance with the guessed results, a specific sort of the operating instruction is selected from a plurality of sorts of the operating instructions, and the selected instruction is displayed as an appropriate instruction for the operator. Further, the one- loop controller for automatizing operation comprises an input interface section for acquiring image information, an image recognition section for recognizing the image using the acquired image information, a control section for calculating control commands according to the image recognition results, and an output interface for outputting control commands to process actuators or subordinate controllers, respectively.

This application is a divisional of application Ser. No. 08/208,584filed Mar. 11, 1994, U.S. Pat. No. 5,619,619.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an information recognition system andcontrol systems using the same recognition system, and more specificallyto an information recognition system for processing large scaleinformation such as image data or speech data that are huge in theinformation quantity and redundant in the representation format.

2. Description of the Prior Art

The conventional information processing system has been so farconstructed by a CPU based upon a Neumann-type computing system usually.Further, the processing speed of the information has been improved byincreasing the processing speed of the CPU itself or by connecting aplurality of CPUs in parallel to each other for simultaneous processing.

In the conventional processing system, however, since the systems is notprovided with means for learning the information recognition criteriafor itself, it has been so far necessary to give the informationrecognition criteria to the processing system previously in the form ofprograms manually. However, in order to recognize large scale data, agreat number of recognition rules must be given to the processing systemwithout inconsistency, with the result that it has been practicallyimpossible to realize the large scale information recognitionprocessing.

On the other hand, it has been well known that there exists a neuralnetwork as an apparatus for learning the recognition criterionautomatically. The neural network is composed of a great number ofneurons. The input and output relationship of each of the neurons isgiven in the form of Zigmond function, for instance. Further, theseneurons are coupled to each other hierarchically via appropriate weightcoefficients adjusted in such a way that the input and output signalrelationship of the whole circuit can be approximated to theinput-teaching data relationship. As the adjusting method, a backpropagation method is well known such that the input data and theteaching data are given repeatedly in order to acquire the recognitioncriteria automatically.

In the conventional neural network, however, the input and output signalrelationship of the whole circuit is learned integrally, in both thecases where the network is realized as software by the computer andwhere the network is realized as hardware by analog or digital circuits.Accordingly, when the large scale data are learned to acquire a greatnumber of recognition rules, since the number of learning repetitionsincreases greatly, it has been also impossible to realize the largescale information recognition processing even with the use of the neuralnetwork.

As described above, the conventional information recognition system isthe Neumann-type recognition system which it is difficult to determinethe recognition rules, or the integral-type neural network which canlearn the recognition rules automatically but is difficult forprocessing the large scale data.

In the case where large scale data are processed by the conventionalinformation recognition system, in order to set the informationprocessing load as low as possible, the basic steps are to select thefeature variables along the object of the final classificationrecognition and to compress the information in accordance with theinformation compression processing represented by the featureextraction. In practice, when appropriate feature variables (some scalerquantities, in many cases) can be selected, it is possible to fairlyreduce the overall processing load thereof.

In the recognition systems actually required in the process control,however, it is not only difficult to always select the feature variableshaving sufficient information required for the final object, but alsothere exists the case where the effective feature variables cannot beobtained under the actual environment of the recognition processingexecution.

To overcome the above-mentioned problem, therefore, various parallelprocessing techniques have been recently adopted in practice with theadvance of the improvement in the computation capability. Among these,in particular, the neural network is expected as an effective method ofclassifying and recognizing large scale data in both the learning withteaching-data type network and the learning without teaching-data typeneural network. The neural network technique has been so far widelyapplied in the fields of character and speech recognition. However, itis reported that the overall recognition rate can be improved bycombination of the conventional recognition processing method with theneural network technique. For instance, there exists a system in which acharacter recognizing apparatus constructed by a conventional Neumanntype computer is combined with a neural network recognizing apparatus.In this system, although the recognition rate of 95% or more isobtained, the neural network is used only to improve the recognitionrate by 2 to 3%.

In the general image recognition, however, it is rather difficult toutilize the conventional recognition processing technique effectively asdescribed above, so that the recognition processing load is oftenincreased markedly on the neural network side. In addition, in the caseof the learning with teaching-data network widely used in the imagerecognition processing, it is very difficult to decide an effectivenetwork architecture in the learning processing (which is importantprocess for realizing the detailed recognition and classificationsystem). That is, there exists such a decisive drawback that thelearning itself cannot be converged abruptly with increasing number ofdata to be classified and/or recognized.

To overcome the above-mentioned problem, it may be possible to adopt thelearning without teaching-data type network (which is less in stagnationof the overall learning processing) as the central network architecture.In the case of the learning without teaching-data type network, althoughbeing effective to the rough category classification, it is oftendifficult to classify or recognize the detailed categories.

On the background as described above, recently, a combination of thenetwork without teaching data and the network with teaching-data hasbeen proposed such that feature extraction processing is executedpreviously to some extent by the network without teaching-data and thenthe output is given to the network with teaching-data for learning ofthe data classification capability.

In the above-mentioned combined system as described above, however, whenthe amount of data to be processed increases hugely, there still existsa problem in that it is difficult to construct a system which cansufficiently recognize and classify the huge data under a practical load(e.g., the number of design items) considered by the current computertechnology.

There has been so far known such a system which is the combination ofthe above two types of networks. This system includes a learning withoutteaching-data type neural network and a plurality of learning withteaching-data type neural networks. In the case of the learning withoutteaching-data type neural network, since the processing required for thearchitecture is only one, exist no wasteful processing. In the case ofthe learning with teaching-data type neural networks, however, since thenetwork processing apparatuses corresponding to the number of theprocessing steps are required (in spite of the network processing in thesame architecture), there exists inevitably a wasteful processing.

In particular, when the neural network processing apparatus is anindividual processing board, the hardware resources required for thesystem construction excessively increase, so that it becomes impossibleto coexist with other systems without any practicability.

Here, in the case of the large scale neural networks having greatnumbers of the inputs and outputs and the intermediate layers areconstructed, if each neural network element is constructed by a singlehardware element one by one, the number of the hardware elements becomeshuge and thereby the number of connections also increasesextraordinarily. As a result, the reliability of the neural networksystem is deteriorated and in addition a very wide space is required forthe hardware.

To overcome the above-mentioned problem, it is possible to construct anapparent neural network composed of a plurality of neural networkelements in the form of software, by controlling the input/outputsignals of the single neural network hardware element in accordance withcomputer software. In more detail, the weight coefficient data betweenthe neural network elements and the input/output signal data of therespective neural network elements are read into a computer through anexternal bus, calculated by a CPU of the computer, and then transferredto the neural network elements through the external bus as theconnection weight data between the neural network elements and theinput/output signal data.

In this method, however, since it takes much time to transfer databetween the neural network and the computer through an external bus, andfurther since the general purpose CPU of the computer does notnecessarily function as an optimum neural network element, a huge timeis required for the large scale neural network learning and the datapropagation in the forward propagation direction. That is the firstpoint of the problem related to the invention described here.

Next, in order to describe an operation of a new data recognition systemto a man-machine interface, some problems of a graphical man-machineinterface apparatus will be described hereinbelow.

For instance, a human face changes in various ways according to hisfeeling or mind. Therefore, it is possible to consider that the humanface includes a lot of useful information. In the conventional graphicalman-machine interface apparatus, however, the operation thereof has beenexecuted irrespective of the expression of the operator's face, that is,the various operator's information. As a result, some problems have beenso far proposed from the user's or operator's standpoint.

One of the above-mentioned problems relates to a cash dispenserinstalled in various banking organizations such as banks or postoffices. In the cash dispenser, the operation required for transferringmoney to another bank is complicated in particular. Although theguidance of operation procedure and inputted contents are displayed on adisplay picture, the displayed instruction is often difficult for someoperators to understand. That is, the operation may be simple and easyfor the persons accustomed thereto. However, this man-machine interfaceapparatus is very troublesome to the person who operates the apparatuson rare occasions or who is poor in handling machines.

In other words, in the conventional man-machine interface apparatus, apredetermined operation sequence is required for the operatorirrespective of the operator's skill. To facilitate the operation, thegraphical user interfaces have been so far widely used. In this case,however, some knowledge over a predetermined level is required for theoperator in advance. Further, there exists an interface apparatus suchthat two different operation sequences are prepared in advance accordingto the operators different in skill so that the operator can select anyone of them. However, this interface apparatus is still notsatisfactory. The reason is as follows: Since the operation is explainedin accordance with a predetermined sequence irrespective of the feelingof the operator now operating the system, the operation sequence is notwell understandable for the non-skilled person so that stress may becaused, or in contrast with this, the skilled person may be irritated.Further, when the apparatus is operated by the non-skilled person, sincethe apparatus workability is lowered, another skilled advisor isnecessary. As described above, when the man-machine interface apparatusis used as machines for selling products, since the machine makes a coolimpression on the user's mind, the man-machine interface apparatus hasbeen so far used for only automatic vending machines.

Next, another problem concerned with man-machine interface apparatuswill be described hereinbelow. Although the above-mentioned problems arewidely noticed in the field of the man-machine interface apparatus,there exists another problem with respect to the person skilled incomputer operation to some extent. The problem is related to theoperation of entering data to the machine through a display picture, inparticular with respect to use of a pointing device. As the pointingdevices, a touch pen, mouse, etc. are so far known. In these pointingdevices, a pointer must be shifted by the operator's hand, so that arelatively large motion is required for the operator whenever thepointer is required to be shifted. For instance, when a cursor on adisplay picture is moved with the use of a mouse, the operator mustfirst take a mouse with his hand, move the mouse on a predeterminedplace (a mouse pad, a desk, etc.), and then click the button on when thecursor is located. In these operation, since some motional actions arerequired for the operator, there exists the case where the cursor cannotbe shifted to a desired position along a considered locus, thus causinga vicious cycle of irritation and erroneous operation, in spite of thefact that a quick operation is required for the operator.

In summary, in the conventional man-machine interface apparatus, sincethe manipulation is not related to the feeling of the operator's face,there exists a problem in that the manipulatability of the interfaceapparatus is not satisfactory.

Next, in order to describe another application of the new datarecognition system to a control system used in some industrial plants,some problems of general control system or controller will be describedhereinbelow.

A one-loop controller has been so far known, by which various operationparameters of a plant can be controlled using image data of an object tobe controlled.

In the conventional one-loop controller, the operation of a plant hasbeen automatized by inputting various measured values such as pressure,flow rates, temperatures, etc. and further controlling the processvariables by operating various actuators with proportional, integral anddifferential calculations as feed-back control action.

In the conventional one-loop controller, however, since onlyone-dimensional information is processed, it has been impossible toadjust the operation parameters of a plant using the operatingconditions observed by the operator; that is, image (two-dimensional)information.

Further, when fuzzy inference is adopted as the control algorithm forthe one-loop controller, the rules and the membership functions used forthe fuzzy inference must be adjusted at the initial setting stage inmany cases. However, since the above-mentioned one-loop controller isnot provided with on-line adjustment functions, great labor is requiredto adjust the rules and the membership functions.

Further, in the conventional one-loop controller, since several-hundredcontrol loops are controlled simultaneously to realize a simplemaintenance, an independent controller is allocated to each controlloop, and these controllers are used as one-loop controller incombination. That is, a plurality of one-loop controllers are mounted ona rack so that a great number of loops can be monitored by a singlepanel. Therefore, the shape of the controller panel is usually narrow inthe width direction and long in the depth direction. Since the frontsurface of the controller panel is narrow, the plant operationparameters (controlled variables, set point values, manipulatedvariables of plants, etc.) are displayed on simple meters arranged onthe control panel, so that a plotter, for instance, is additionallynecessary to check the time trends Further, even if monitor screens areprovided in the front surface of the controller panel, the displayedpicture surfaces are narrow, so that the manipulatability is low.

Further, in the case of the visual feedback control based upon imagerecognition and image information with the use of the neural network, itis necessary to first execute the learning operation of the neuralnetwork by use of a great number of teaching data. A problem which hasarisen in the conventional feedback control will be explained. The imagerecognition by use of the neural network can be executed in accordancewith the following procedure: First, features are extracted from a greatnumber of learning image data, and the image data are classifiedaccording to several categories on the basis of the extracted features(in the case of image data having the same features with respect toseveral evaluation criteria). Thereafter, a great number of learningimage data are inputted to the neural network, and the neural networklearning is executed until the outputs of the neural network can beroughly equalized to each other for the inputted learning image databelonging to the same category. In other words, since the imagerecognition precision and the convergent speed of the neural network aredependent upon the classification precision of the learning image data,when the feature extraction and the classification method of thelearning data are not appropriate, it is impossible to allow the neuralnetwork to learn image data, so that the image recognition itself of theneural network is deteriorated. Conventionally, when the image data arerecognized by the neural network, a skilled operator observes image dataone by one independently and compares the observed image data withappropriate data at need in order to classify the image data intoseveral categories. In this case, however, since the number of imagedata required for the neural network is huge, so that it is extremelydifficult for the skilled operator to classify the image data on thebasis of universal criteria. As a result, there exists many cases wherethe image data having the similar features are classified into othercategories, so that it has been difficult to improve the image datalearning efficiency and the image recognition precision.

There has been another problem arisen in the conventional feedbackcontrol. When the shape recognition is executed using image informationand the neural network learning, since there exist no indices forsetting appropriate initial values of the coupling weights between themutual nodes of the neural network, the initial values have been so fardetermined on the basis of random numbers. However, a long time isrequired for such an initial learning that the coupling weights must betuned until a constant effect can be obtained after the random numbershave been set, so that the efficiency is low. In addition, when theshape recognition is executed on the basis of image information of lowS/N ratio, the outputs of the neural network are unstable, so that thesystem reliability is low.

SUMMARY OF THE INVENTION

With these problems in mind, therefore, it is the object of the presentinvention to provide an information recognition system which can processthe large scale information stably. To achieve the above-mentionedobject, the present invention provides an information recognitionsystem, comprising: a plurality of recognition processing units eachcomposed of a neural network; a plurality of teacher signal transmissionlines for supplying teacher signals to each of a plurality of saidrecognition processing units, individually; a plurality of processeddata transmission lines for supplying data to be processed to each of aplurality of said recognition processing units, individually; aplurality of output signal transmission lines responsive to the teachersignals and the processed data, for outputting output signals to each ofa plurality of said recognition processing units, individually; andmeans for connecting each of a plurality of said output signaltransmission lines to a plurality of said processed data transmissionlines, respectively.

Further, the present invention provides an information recognitionsystem, comprising: a category classification apparatus for roughlyclassifying data to be processed by a plurality of stages of learningwithout teacher-data type neural networks; and a category recognitionapparatus for finally recognizing the data to be processed by a learningwith teacher-data type neural network, for each category classificationdecided as a final stage output of said category classificationapparatus.

Further, the present invention provides an information recognitionsystem, comprising: first recognition processing means composed of atleast one learning without teacher-data type neural network, functionsof said first recognition processing means being decided whenpredetermined construction information for said learning withoutteacher-data type neural network has been set; second recognitionprocessing means composed of at least one learning with teacher-datatype neural network, functions of said second recognition processingmeans being decided when predetermined construction information for saidlearning with teacher-data type neural network has been set; firststoring means for storing the construction information of theteacher-absent and learning with teacher-data type neural networks;second storing means for storing the information to be processed by saidfirst and second recognition processing means; and control means forexecuting a plurality of sorts of recognition processing by switchingsaid first and second recognition processing means and switching thesetting information stored in said first and second storing means.

According to the information recognition system of the presentinvention, since a plurality of neural networks for sharing thepredetermined functions, respectively can be learned individually andsince an overall recognition processing can be achieved in combinationof these neural networks, it is possible to realize the informationrecognition system which can automatically acquire a great number ofrecognition rules on the basis of a great number of learning data, whichhas been so far not realized. Further, the number of iterativecalculations required for acquiring rules can be reduced down to apractical number. In addition, since each neural network unit can becomposed of about several tens of neurons in general, the number of thecoupled circuits can be reduced down to a realizable number. Further,since the coupling circuits between the units can be composed of theinput and output signal lines between the respective units, the numberof the circuits is realizable. Therefore, it is possible to solve theproblem in that the number of the coupling circuits increasesdrastically, which has been so far involved in the conventional largescale neural network system.

Further, in the case where the category classification by the clusteringprocessing is executed at a plurality of stages, as far as the number oflearning data belonging to the finally classified categories issufficiently reduced, it is possible to roughly secure the convergenceof the learning by use of the category classification recognition systemprovided with a hierarchical neural network architecture. Therefore, itis possible to construct the recognition system under the practicalengineering load and further to realize appropriate high-speedrecognition and classification for a large scale data group.

Further, in the case of the conception such that a limited number ofnetwork hardware resources can be used as a great number of networks byswitching data inputted to the network hardware resources, theprocessing executed by a great number of neural networks can be realizedby a minimum possible number of practical processors, thus allowing thelarge scale data recognition to be applicable to various fields inpractice. Further, in the case of a single-chip neural network system,since a large scale neural network system can be realized withoutincreasing the number of neural network hardware elements, the spacerequired for the neural network system and the number of the connectionwires can be reduced. Accordingly, the reliability of the neural networksystem can be improved. Further, since the data can be transferredbetween the neural network hardware elements and the neural networkstoring RAM through the internal bus constructed within the same chip,the data can be transferred at high speed, so that it is possible toshorten the calculation time required for the neural network system. Inaddition, since these elements can be constructed on the same chip, itis possible to improve the reliability of the hardware elements.

Further, another object of the present invention is to provide agraphical man-machine interface system which can extract usefulinformation from the image data indicative of an operator's face andcontrol the operating display instruction on the basis of the extractedinformation to display an operating instruction suitable for theoperator.

To achieve the above-mentioned object, the present invention provides aman-machine interface system, comprising: image sensing means forobtaining image signals indicative of an operator's face; imagerecognizing means for extracting factors related to operator facialexpressions from the image signals obtained by said image sensing means,recognizing operator feeling from the extracted factors, and outputtingrecognition results as image recognition signals; a display apparatus;instruction data storing means for storing a plurality of sorts ofmachine operating instruction data; and display control means forselecting specific sort of the machine operating instruction data from aplurality of sorts of the machine operating instruction data accordingto the image recognition signals, and for controlling the displayapparatus to display the selected instruction data as the machineoperating instruction suitable for the operator.

Further, the present invention provides a man-machine interface system,comprising: image sensing means for obtaining image signals indicativeof an operator's face; image recognizing means for extracting factorsrelated to operator facial expressions from the image signals obtainedby said image sensing means, recognizing operator's eye positions on adisplay picture by the extracted factors, and outputting recognitionresults as image recognition signals; displaying means; and displaycontrol means for displaying a pointer on said displaying meansaccording to the image recognition signals indicative of the operator'seye positions.

In the man-machine interface system according to the present invention,since the skillfulness of the operator can be discriminated on the basisof the image signals and further since an appropriate displayinstruction can be selected according to the operator's skillfulness, itis possible to realize the interface system suitable for the operator.

Further, in the man-machine interface system according to the presentinvention, since the eyes of the operator are detected by the imagesignals and further since the pointer is controllably moved according tothe operator's eyes, it is possible to move the pointer freely on thedisplay picture whenever the operator moves his face or eyes, so that itis possible to realize a quick and comfort operation.

Further, the other object of the present invention is to provide aone-loop controller for automatizing process operation, and an apparatusfor displaying image data and an apparatus for inputting neural networkconnection weights, both suitable for the one-loop controller.

To achieve the above-mentioned object, the present invention provides aone-loop controller for feedback controlling operation, related toshapes of an object to be controlled using image signals indicative ofthe object, comprising: an input interface section for converting anobserved image of the object to be controlled into image data; an imageprocessing section for extracting a plurality of sorts of featureparameters of the observed image from the extracted image data; arecognition section for recognizing the observed image using the basisof a plurality of sorts of the feature parameters; a control section forgenerating control commands on the basis of the recognized results ofthe image; and an output interface section for outputting the controlcommands to controllers for the process.

Further, the present invention provides a one-loop controller,comprising: simulating means for simulating control operation foradjusting control parameters using an observed image indicative of anobject to be controlled; means for displaying the observed imageindicative of the object to be controlled; means for acquiring imagedata at predetermined times or predetermined time intervals; means forstoring the acquired image data; and means for displaying the storedimage data of a predetermined range on said displaying means as sampledimage data to be given to said simulating means.

Further, the present invention provides a one-loop controller,comprising a plane image display device accommodated on one of left andright sides of and within said controller so as to be drawn frontwardaway from the controller.

Further, the present invention provides an image data displayingapparatus for the visual feedback controller, comprising: means fordisplaying a plurality of image data on a display picture; means forclassifying the image data into categories determined on the basis ofsome evaluation criteria; and means for simultaneously displaying aplurality of image data belonging to the categories classified by saidclassifying means on said displaying means.

Further, the present invention provides a neural network coupling weightinputting apparatus for the visual feedback controller, comprising:means for inputting connection weights at a plurality of nodes of aninput layer of a neural network for recognizing an image pattern nodesbetween a plurality of nodes of intermediate layers of the same neuralnetwork, to the neural network; means for displaying an image to becontrolled on a display picture by dividing an image region into aplurality of regions according to the nodes of the input layer of theneural network; means for designating at least one of the nodes of theintermediate layers of the neural network; means for determining theconnection weight by designating a divided area on the display pictureby the designating means, to input the connection weight to the neuralnetwork.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a block diagram showing a neural network board of anembodiment of the information recognition system according to thepresent invention;

FIG. 1B is an enlarged block diagram showing the same neural networkboard shown in FIG. 1A;

FIG. 2 is a block diagram showing an image recognition apparatus inwhich the neural network boards as shown in FIGS. 1A and 1Bare,incorporated;

FIGS. 3A, 3B and 3C are illustrations showing an example of theoperation of the apparatus shown in FIG. 2;

FIG. 4 is a block diagram showing the basic configuration of theapparatus for enabling category classification;

FIG. 5 is a flowchart for assistance in explaining the learningprocessing of the apparatus shown in FIG. 4;

FIGS. 6A and 6B are illustrations for assistance in explainingeach-stage cluster designation processing by use of a pointing device inthe construction processing of the learning without teacher-data typeneural network which constitutes the category classification apparatus;

FIG. 7 is an illustration for assistance in explaining theclassification processing of the learning without teacher-data typeneural network which constitutes the category classification apparatus;

FIG. 8 is an illustration for assistance in explaining the learning withteacher-data type neural network architecture which constitutes thecategory-classified recognition apparatus;

FIG. 9 is a flowchart for assistance in explaining the recognitionprocessing of the apparatus shown in FIG. 4;

FIG. 10 is a block diagram showing another modification of theinformation recognition system according to the present invention;

FIGS. 11A, 11B, 11C, 11D, 11E, and 11F are illustrations showing variousexamples of the char bed shapes;

FIG. 12 is a block diagram showing a system architecture, in which thesystem shown in FIG. 10 is applied to the char bed shape recognitionprocessing in a recovery boiler plant;

FIG. 13 is a block diagram showing a practical construction of theinformation recognition system adopted as a core section of the systemshown in FIG. 12;

FIG. 14 is a block diagram showing another practical construction of therecognition system adopted as a core section of the system shown in FIG.12;

FIG. 15 is a block diagram showing the other practical configuration ofthe information recognition system adopted as a core section of thesystem shown in FIG. 12;

FIG. 16 is an illustration showing a processing image of the recognitionoperation of the learning without teacher-data type neural network;

FIG. 17 is an illustration showing a processing image of the recognitionoperation of the learning with teacher-data type neural network;

FIG. 18 is an illustration showing a processing image of the learningoperation of the learning with teacher-data type neural network;

FIG. 19 is a block diagram showing an example of the hardwareconstruction of the system shown in FIG. 10;

FIG. 20 is a flowchart for assistance in explaining theforward-direction propagation control algorithm of the system shown inFIG. 19;

FIG. 21 is a flowchart for assistance in explaining thereverse-direction propagation control algorithm of system shown in FIG.19;

FIG. 22 is an outside view showing a first embodiment of the man-machineinterface system according to the present invention;

FIG. 23 is a block diagram showing the system shown in FIG. 22;

FIG. 24 is an illustration for assistance in explaining the operation ofthe system shown in FIG. 22;

FIG. 25 is a block diagram showing the hardware construction of apointing device system as a second embodiment of the man-machineinterface system according to the present invention;

FIG. 26 is a flowchart for assistance in explaining the pointer controlprocessing of the system shown in FIG. 25;

FIG. 27 is a flowchart for assistance in explaining the detailed eyeposition detecting processing shown in FIG. 26;

FIG. 28 is a block diagram showing a first embodiment of a one-loopcontroller according to the present invention;

FIG. 29 is a block diagram showing an embodiment of the imagerecognition section of the controller shown in FIG. 28;

FIG. 30 is a block diagram showing a multilayer neural network;

FIG. 31 is graphical representation showing membership functions of aconclusion section;

FIG. 32 is a block diagram showing a second embodiment of a one-loopcontroller according to the present invention;

FIG. 33 is a functional block diagram showing the control parameteradjusting function according to the present invention;

FIG. 34 is an illustration showing an embodiment of an operation displaypicture for realizing the control parameter adjusting function;

FIG. 35 is an illustration showing an embodiment of an operation displaypicture for realizing the rule correcting function;

FIG. 36 is an illustration showing an embodiment of an operation displaypicture for realizing the membership function correcting function;

FIG. 37 is a block diagrams showing a third embodiment of the one-loopcontroller according to the present invention;

FIG. 38 is a functional block diagram showing the connection weightlearning function shown in FIG. 37;

FIG. 39 is an illustration showing an embodiment of an operation displaypicture for realizing the weight coefficient learning function;

FIG. 40 is an illustration showing an embodiment of an operation displaypicture for realizing the teacher data input function;

FIG. 41 is a block diagram showing a one-loop visual feedback controlleraccording to the present invention;

FIG. 42 is a block diagram showing a simulator section of the controllershown in FIG. 41;

FIG. 43 is an illustration showing a menu picture for forming imagedata;

FIG. 44 is an illustration showing a menu picture for controlsimulation;

FIGS. 45A and 45B are illustrations showing a one-loop controllerprovided with a drawer type display panel arranged on the inner sidesurface of a rack;

FIG. 46 is an illustration showing an example of installing a pluralityof one-loop controllers;

FIG. 47 is an illustration showing an example of the one-loop controllerhaving a display panel on a side surface of a rack;

FIG. 48 is an illustration showing an example of the display picture ofthe image data displaying apparatus;

FIG. 49 is a block diagram showing a hardware construction of the imagedata display apparatus;

FIG. 50 is an illustration showing an example of the monitor picture ofan object to be controlled;

FIG. 51 is an illustration showing an example of the monitor picture, inwhich the image is divided into small regions by inserting meshesthereon;

FIG. 52 is an illustration showing an example of the display picture, inwhich a window 52d is displayed to input the number of intermediatelayer units;

FIG. 53 is an illustration showing an example of the display picture, inwhich the connection weights are set by grouping the divided images ofan object to be controlled;

FIG. 54 is an illustration showing an example of the display picture, inwhich warning is indicated when the number of shapes to be recognized issmaller than the number of the intermediate layer units; and

FIG. 55 is an illustration showing an example of the display picture, inwhich coupling weights are displayed.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The embodiments of the information recognition system according to thepresent invention will be described hereinbelow with reference to theattached drawings.

The information recognition system according to the present inventionshown in FIGS. 1A and 1B is constructed by a plurality of neural networkunits NN. Each neural network unit NN can execute learning individuallyon the basis of learning data INs and teaching data INt inputted theretoindependently.

A change-over switch SW is connected between the respective units NN.The output terminal OUT of each unit NN is connected to one inputterminal b of the switch SW, and learning data INs are supplied to theother input terminal a of the switch SW.

FIG. 1B is an enlarged view showing each unit NN and each switch SW.FIG. 1A shows only one input line and one output line and one switchcorresponding to each unit for brevity. In practice, however, as shownin FIG. 1B, each unit NN is provided with a plurality of input andoutput lines, and the same number of switches SW are provided so as tocorrespond to the input and output lines of each unit NN, respectively.

In learning, each change-over switch SW is set to the terminal a, sothat learning data INs and teaching data INt are given to each unit NN.Each unit NN keeps learning until the respective output signals of eachunit converge independently. The output signals of each unit areoutputted from the output terminals OUT via the terminals b of theswitches SW, so that it is possible to check the learning status. Uponconvergence of the respective units NN, all the change-over switches SWare set to the terminals b, so that it is possible to construct a largescale neural network board composed of combinations of the respectivefunctions. When respective data IN₁₁, IN₁₂ and IN₁₃ to be recognized aregiven to the respective first-stage units NN₁₁ NN₁₂ and NN₁₃ of theneural network constructed as described above, these units recognize thegiven data in accordance with the learned recognition rules. Therecognition results outputted from the output terminals OUT₁₁, OUT₁₂ andOUT₁₃ of the units NN₁₁ NN₁₂ and NN₁₃ are given to respectivesecond-stage units NN₂₁ NN₂₂ and NN₂₃ via the switches SW₁₁, SW₁₂ andOUT₁₃. Further, these recognition results of these units are furthergiven to respective third-stage units NN₃₁ NN₃₂ and NN₃₃ via theswitches SW₂₁, SW₂₂ and OUT₂₃, respectively, so that the finalrecognition results are outputted from the output terminals OUT₃₁, OUT₃₂and OUT₃₃ of the units NN₃₁ NN₃₂ and NN₃₃, respectively.

Further, in the above-mentioned description with reference to FIGS. 1Aand 1B, although the input and output lines and the change-over switcheshave been described as hardware representation, in practice theseelements may be constructed as software on a computer.

FIG. 2 shows an image recognition control apparatus, to which the neuralnetwork board constructed as described above is applied.

In FIG. 2, the control apparatus first obtains an image signal detectedby a TV camera 101. Since being of NTSC signal, the obtained imagesignal is converted into digital signals by an image signal acquisitionboard 102. On the basis of the converted digital signals, an imageprocessing board 103 extracts the features of an object to berecognized. The features are determined by detecting the edges of apicture, brightness of the whole picture, the gravity center of anobject to be image sensed, etc. On the basis of the signals of the imageprocessing board 103, a large scale neural network board 104 recognizesand decides the object in accordance with the learned recognition rules.The recognized results are outputted to a control calculation processingboard 105. The control calculation processing board 105 outputs acontrol signal in accordance with a predetermined logic.

The operation of the image processing board 103 and the large scaleneural network board 104 will be described hereinbelow with reference toFIGS. 3A to 3C.

FIG. 3A shows an original picture taken by the TV camera 101. Beingslightly deteriorated by the digitization processing, the image signalequivalent thereto is inputted to the image processing board 103.Processing the received image signal, the image processing board 103outputs a picture as shown in FIG. 3B, when an edge detection isinstructed to the image processing board 103. On the basis of thepicture outputted by the image processing board 103 as shown in FIG. 3B,in the case where the large scale neural network board 104 learns thedecision of a mountain height for instance, the neural network board 104outputs a display of "high mountain"(which is a decision result) asshown in FIG. 3C. The above-mentioned decision results can be changedfreely in dependence upon the learning data and the teaching data bothobtained in the learning step.

With the use of the decision function of the neural network board 104,it is also possible to decide human facial expression or to analogizehuman feeling. In this case, the following learning are executed forinstance: eyes, eyebrows and a mouth are extracted on the basis of thewhole face image data in order to decide the human facial expression.

In more detail, in the respective units NN as shown in FIG. 1A (whichcorrespond to the neural network board 104 in FIG. 2), the roles areallocated to the respective units NN, for instance as follows: theneural network unit NN₁₁ extracts eyes from the whole face picture;NN₂₁, corrects the positions and sizes of the extracted eyes inaccordance with the recognition rules; NN₃₁ detects a tense eye pattern;NN₁₂ extracts eyebrows from the whole face picture; NN₂₂ corrects thepositions and sizes of the extracted eyebrows; NN₃₂ detects a tenseeyebrow pattern; NN₁₃ extracts a mouth from the whole face picture; NN₂₃corrects the positions and sizes of the extracted mouth in accordancewith the recognition rules; NN₃₃ detects a tense mouth pattern,respectively.

In this case, the image data representative of the whole face are givenas the learning data for the unit NN₁₁, and the data indicative of theeyes are given as the teaching data. Further, every possible patternsare given and learned until the image signal output indicates the eyes.By the above-mentioned learning, the unit NN₁₁ connotes the overallfeature detecting capability including the eye shape and the positionalrelationship between the eye shape and the other parts in the face, sothat it is possible to specify the eyes from the whole face image, inthe same way as with the case where a man can judge the presence of theeyes by seeing the face.

As the learning data of the unit NN₂₁, various patterns indicative ofeyes of various sizes existing at various positions (i.e., variouspatterns expected to be outputted from the unit NN₁₁ when the system isused in practice) are given. As the teaching data, the patternscorresponding thereto and further corrected in position and size aregiven. The learning is repeated on the basis of the given learning andteaching data. Accordingly, the unit NN₂₁ can output soon an eye patternobtained by correcting the eyes of various sizes existing at variouspositions to the eyes of a constant size existing at constant positions.

As the learning data of the unit NN₃₁, various size eye patterns of adetermined eye size existing at determined positions (i.e., variouspatterns predicted to be outputted by the unit NN₂₁,) are given, and thedata indicative of whether the corresponding pattern is a tense pattern((1) Positive) or not ((0) Negative) are given as the teaching data. Thelearning is repeated until the answer of (1) or (0) can be obtained at aconstant rate (i.e., until a correct answer can be obtained except thecase where it is difficult for a man to discriminate whether the eyesindicate tense feeling or not).

The same as above is applied to the case of the eyebrows and mouth. Insummary, the image data (learning data) indicative of the whole face andthe image data (teaching data) indicative of the eyebrows are given tothe unit NN₁₂. Further, the eyebrow patterns (learning data) of varioussizes existing at various positions and the eyebrow patterns (teachingdata) of a predetermined size existing at predetermined positions aregiven to the unit NN₂₂. Further, the various eyebrow patterns (learningdata) of various shapes of a predetermined size existing atpredetermined positions and the data indicative of whether thecorresponding patterns is a tense pattern ((1) Positive) or not ((0)Negative) are given to the unit NN₃₂ for respective learning.

In the same way, the image data (learning data) representative of thewhole face and the image data (teaching data) indicative of the mouthare given to the unit NN₁₃. Further, the mouth patterns (learning data)of various sizes existing at various positions and the mouth patterns(teaching data) of a predetermined size existing at a predeterminedposition are given to the unit NN₂₃. Further, the various mouth patterns(learning data) of various shapes of a predetermined size existing at apredetermined position and the data indicative of whether thecorresponding patterns is a tense pattern ((1) Positive) or not ((0)Negative) are given to the unit NN₃₃ for respective learning.

Further, after the respective learning have been converged, therespective change-over switches SW (i-1)j are set to the b side. In thepractical use, it is possible to obtain a final decision owing to thelinking functions of the respective units NN_(ij). As described above,in the large scale information recognition system of the presentinvention, it is possible to acquire a great number of recognition rulesautomatically on the basis of a great number of learning data, whichhave been so far not realized. Further, the number of the repetitivecalculations required to acquire the rules is an actual computationnumber. Further, since each of the neural network units is composed ofseveral tens of neurons in usual, the number of the coupling circuits isa realizable number. Further, since each of the coupling circuitsbetween the units is also composed of buses of input signals and outputsignals applied to and from each unit, the number of the circuits is aneasily realizable number, so that it is possible to prevent the numberof the coupling circuits from being increased drastically (which hasbeen so far involved in the conventional neural network circuit).

Another embodiment of the information recognition system according tothe present invention will be described hereinbelow.

The final object of the information recognition system is to recognizeor classify data to be processed on the basis of some featuredcharacteristics included in the data themselves. However, in the casewhere the feature variable index which can represent the featuredcharacteristics effectively in the form of compression is not knowndefinitely, the classification is to be featured by the representativedata for each category to be classified. In other words, it is necessaryto construct a system which can designate essential characteristics onthe basis of the representative data collected for each category andfurther classify the newly obtained data into appropriate categoriesappropriately.

Therefore, the recognition system on the basis of categoryclassification will be described with reference to FIG. 4. First,information recognition system of the present embodiment is roughlycomposed of a category classifying apparatus 401, a neural networkswitching apparatus 402, and a category recognizing apparatus 403. Thecategory classifying apparatus 401 roughly classifies data to beprocessed by plural stages of learning without teaching-data type neuralnetworks. For instance, in the case of the character recognition ofhand-written Japanese cursive (hiragana) characters, Chinese charactersand Japanese syllabary (katakana), the category classifying apparatus401 recognizes as to which character attribute the character to berecognized belongs to. The category recognizing apparatus 403 isprovided with a plurality of teaching-present neural networks 4031,4032, . . . 403n each having a function for recognizing each kind ofcharacters (e.g., in the case of Japanese cursive characters, any one of(a), (j), (u), . . . in the order of Japanese alphabetical order)determined for each character attribute of these characters. The neuralnetwork switching apparatus 402 selects the neural network to beactivated in accordance with the category data outputted by the finalstage of the category classifying apparatus 401, and suppliesinformation to be processed to the selected neural network.

FIG. 5 is a flowchart showing the procedure of the category classifyingapparatus 401 and the category recognizing apparatus 403.

In FIG. 5, first in step 201, the learning data corresponding to thecluster at a stage are classified by the learning without teaching-datatype neural network. Here, if the current classification stage isdenoted by i, in step 201, one of a plurality of clusters at the (i-1)stage is classified. Successively, in step 202, control checks whetherthe classification of all the clusters has been completed or not, inother word, whether there still exists some clusters not yet classified.In the case where a cluster not yet classified exists, control returnsto step 201. As described above, the clusters at the first stage arerepeatedly classified in steps 201 and 202. Further, as the results ofthe step 202, when all the clusters at the stage to be classified havebeen classified, control proceeds to step 203 to check whether thenumber of learning data corresponding to the new cluster is appropriateor not. In other words, control checks whether the clusterclassification corresponds to the learning data to be processed by thelearning with teaching-data type neural network in the later process andfurther whether the number of the data is appropriate as the number ofthe learning data, and in addition whether the number of the clustersexceeds the number of the categories to be classified or not or whetherthe number of learning data is excessively large or not. As the resultsof above-mentioned judgements, if the number of the learning data is notappropriate, for instance, control proceeds to step 206 to set furthersuccessive stages, returning to step 201.

As described above, the control steps 201 and 203 are repeatedlyexecuted until the clusters can be classified into a number of clustersappropriate to the number of learning data for the learning withteaching-data type neural network. That is, a great number of learningdata given as representative data for each category are to be classifiedinto some clusters at each stage of the category classifying apparatus401 by the learning without teacher-data type neural network. As thelearning rules of the learning without teaching-data type neuralnetwork, it is possible to utilize the learning method represented byLVQ (Learning Vector Quantization) method (by T. Kohonenn:Self-Organization and Associative Memory (3rd Ed.), pp 199 to 209,Springer-Verlag (1989)) or other related and improved methods. To decidethe clusters, the algorithm so far proposed can be used as it is.However, it is also possible to allow the system constructor (operator)to have charge of the cluster decisions. In particular, in the casewhere the process to be recognized includes some process executed by aperson in usually, it is necessary to allow the system constructor to bepositively related to the clustering processing in order to construct adesirable system. FIGS. 6A and 6B are illustrations showing the statuswhere the cluster can be set by use of a pointing device such as a mouseor a touch pen. When an area is designated by a mouse pointer as shownin FIG. 6A, the data within the designated area are registered as thesame cluster as shown in FIG. 6B. As described above, when the clusteris determined freely by use of a pointing device, it is possible toeffectively classify the categories on the basis of the wholerecognition processing.

At the respective stages other than the first processing by the categoryclassifying apparatus 401 as shown in FIG. 4, only the learning databelonging to each cluster are further classified for each cluster by thelearning without teaching-data type neural network. FIG. 7 shows thestatus where the clusters are classified at a plurality of stages. Byrepeating the above-mentioned cluster classification, it is possible tofinally obtain a tree structure in the cluster classification for thelearning data through the processing by the learning withoutteacher-data type neural network. In FIG. 7, for instance, theclassification address for the data αis given by (2, 4, . . . , 1) (i-thelement denotes the belonging cluster number at the i-th stage),respectively. Accordingly, the category classifying apparatus 401constructed by the classification functions of the learning withoutteacher-data type neural networks of a plurality of stages inputs dataand outputs the classified categories in the form of directory pathinformation of the belonging clusters at the final stage.

Further, the learning data for each classified category (whose numbercorresponds to the number of the directory paths) obtained at the finalstage are learned with respect to the detailed correspondencerelationship to the final recognition results in accordance with steps204 and 205 shown in FIG. 5. That is, in step 204, single learning datais classified in detail. Further, after the data has been converged, instep 205 control checks whether there are non-processed learning data ornot. As the results, if non-processed learning data still exist, controlproceeds to step 204, and one of the remaining learning data is learned.As described above, when all the learning data have been classified indetail, the system processing ends.

FIG. 8 shows an architecture image of the respective neural network ofthe category recognizing apparatus 403 shown in FIG. 4. The input andoutput form is the same as the learning form of the ordinaryhierarchical structure type neural network. The input data are the imagedata (the input units for all pixels can be prepared, or it is possibleto compress the input information in some way), and the output data arethe recognized results of the input data (the output units can beprepared for each recognition or classification item in the same way asthe input, or it is possible to code the output information in someway). As the learning rules, it is possible to utilize a method referredto as back propagation method as disclosed in a document by Rumelhart(Nature, vol. 323. pp 533 to 536, 1986) or other related and improvedmethods.

As described above, when the learning data increase, the convergencespeed of the learning with teacher-data by the hierarchical architecturetype neural network is reduced markedly. Therefore, in order toappropriately decide the parameters (e.g., the number of network layers,the number of units for each layer, the initial values of the couplingweights thereof, etc.) for determining the convergence performance, aserious problem equivalent to the complicated non-linear optimizationproblem arises, so that it is difficult to obtain a practicalconvergence performance under practical loads. In the system of thepresent invention, however, since the categories are classified by theclustering processing of a plurality of stages, as far as the number ofthe learning data belonging to the respective finally classifiedcategories is reduced sufficiently, the learning by the categoryrecognizing apparatus 403 provided with the hierarchical architecturetype neural network is securely converged, so that it has becomepossible to construct a recognizing system under a practical engineeringload. Further, after the learning for all the categories has beencompleted by the category recognizing apparatus 403, the recognitionsystem can be constructed completely.

The recognition processing (as shown by the recognition processingflowchart in FIG. 9) of data which are not the learning data is roughlyequivalent to the system construction procedure (from which the learningprocess is excluded). In more detail, in step 301, when the data to berecognized are inputted to the category-classifying-apparatus 401 asshown in FIG. 4, the category classifying apparatus 401 outputs thefinally classified category of the given data and the data themselves tothe recognizing neural network switching apparatus 402. Or else, thecategory classifying apparatus 401 outputs data indicative of absence ofthe consistent path to the neural network switching apparatus 402.Successively, in step 302, control checks whether there exist aconsistent classification path in the category classifying apparatus401. As the results, if it exists, control proceeds to step 303, inwhich the neural network switching apparatus 402 transmits the data tobe recognized to the category recognizing apparatus 403 corresponding tothe received final classified category. Further, in step 304, thecategory recognizing apparatus 403 outputs the finally recognizedresults. In step 302, when control determines that there exists noconsistent classification path, a recognition disabling signal ismonitored in step 305, ending the control processing.

A series of the above-mentioned recognition processing is small incomputation burden. Therefore, when limited to only the recognizingprocessing, it is possible to realize a practical recognition system forlarge scale data by use of a computer (e.g., a personal computer) with arelatively small computing performance.

Further, in the information recognition system of the present inventionas described above, it is possible to realize appropriate high-speedrecognition and classification of large scale data groups, with the useof the recognition and classification system provided with the learningwithout teacher-data type neural network for executing categoryclassification at a plurality of stages and with the learning withteacher-data type neural network for executing detailed recognition andclassification.

Another modification of the information recognition system according tothe present invention will be described hereinbelow.

First, in FIG. 10, when both a learning without teaching data typearchitecture and a learning with teaching-data type architecture arerequired for a plurality of neural network processing, the system forexecuting the practical processing must be provided with exclusiveprocessing apparatuses corresponding to each architecture (i.e., alearning without teaching-data type network processing apparatus 801 anda learning with teaching-data type network processing apparatus 802).The individual neural network processing begins when a network managingapparatus 803 commands a network-switching-apparatus 806 to switch theprocessing operation to an individual processing apparatus provided withthe network architecture required for the processing. In selection of anetwork, the network-managing-apparatus 803 further commands aprocessed-data-storing-apparatus 804 and anetwork-architecture-data-storing-apparatus 805 to transfer data to beprocessed and architecture data (connection weight data, etc.) of thenetwork required for the processing, to a neural network processingapparatus (the learning without teaching data type network processingapparatus 801 or a learning with teaching-data type network processingapparatus 802). Owing to the transfer command, the processing data andthe network coupling weight data are transferred to the neural networkprocessing apparatuses 801 and 802 and the internal memory units 807 and808 of the processing apparatuses, respectively. When the processingexecution is enabled, the neural network processing apparatus 801 or 802informs the network managing apparatus 803 of a completion of processingpreparation. In response to an answer signal of processing executionstart, the neural network processing apparatus executes predeterminedprocessing, writes the execution results in the processed data storingapparatus 804, and transmits a processing end signal to the networkmanaging apparatus 803, thus completing one neural network processing.By repeating the above-mentioned processing, it is possible to realize agreat number of neural network processing with the use of the minimumnumber of exclusive network processing apparatuses which correspond tothe number of the network architectures required for the processing.

The above description has been made in case of the processing by thedeterministic networks. In the case where the neural network processingapparatuses 801 and 802 are provided with a function of learning,respectively, however, the processing data (the learning data in thiscase) are read for each learning-process in the same way as with thecase of the above-mentioned processing. On the other hand, the initialvalues of the network architecture data (connection weight data) areread once at the processing start, and thereafter updated in the storageform in the respective memory unit (807 or 808) of the respectiveprocessing apparatus (801 or 802). After the processing end, theconnection weight data are written in the network architecture datastoring apparatus 805, instead of that the processing results arewritten in the processing data storing apparatus 804.

An embodiment of the information recognition system according to thepresent invention will be described hereinbelow, in which therecognition system is applied to a system for recognizing char-bedshapes within a recovery boiler furnace.

The recovery boiler is used in a paper-pulp plant. In the recoveryboiler, organic constituents contained in a waste liquid (referred to asblack liquor) produced in chip cooking (or digesting) process are burntwithin a boiler to form vapor by the combustion heat. Further, costlychemicals (used for the cooking process) contained in the waste liquidis recovered by the utilization of the chemical reaction produced in thesediment (referred to as char-bed) of the dried waste liquid existingwithin the boiler. In the recovery boiler, since the operating conditionchanges markedly according to the characteristics of the black liquor orthe progress of the chemical reaction, it is difficult to automate theoperation of the recovery boiler perfectly, so that the recovery boilerhas been so far operated by some expert operators who always monitor thefurnace conditions. In this case, the operators judge the furnaceconditions with reference to the shapes of the char-bed as shown inFIGS. 11A to 11E. Therefore, a system for automatically recognizing thechar-bed shapes is indispensable for the automation of the recoveryboiler system. In practice, however, the images displayed on monitorsand watched by the operators are not limited to only clear images asshown in FIGS. 11A to 11E. Rather, in general, there are many caseswhere the outline of the char-bed mountain is not clear as shown in FIG.11F or there exists the case where the dust-like substances are oftenattached to an edge of camera lenses for detecting the interior of thefurnace.

In case of the image processing such that the ordinary edge detectionprocessing and the binary processing are executed in order to obtain thechar-bed contour information from the original images, the processedimages include many images of the dust attached to the edges in thecamera visual field or caused by background noise within the furnace.Therefore, in case of the conventional image recognition technique, itis extremely difficult to realize a stable recognition free from aserious error, which has been so far executed by expert operators (whooperate the boiler unconsciously very well).

Therefore, in order to realize as a robust and stable recognitionfunction as a human operators, that is, in order to realize a stablerecognition according to various changes in the furnace combustionconditions (caused by changes in various operating points), it isnecessary to learn large number of image data set, which may amount toseveral hundred data at the learning stage. Accordingly, it is requiredto incorporate various classification processing with the use of manylearning without teaching-data type networks and many learning withteaching-data type networks to realize the architecture of therecognition system as described above.

FIG. 12 shows an example of the system architecture for realizing thechar-bed recognition as described above.

In more detail, the char-bed image acquired by a furnace camera 902 fordetecting the inside of a recovery boiler 901 is pre-processed (i.e.,noise reduction, etc.) by a pre-processing apparatus 904, and thentransmitted to a processed data storing apparatus 804 (which correspondsto the processing data storing apparatus 804 shown in FIG. 10) of a datarecognition apparatus 905 (which corresponds to the system shown in FIG.10). In the data recognition apparatus 905, as already explained, therespective processing boards of the learning without teaching-data typeneural networks and the learning with teaching-data type neural networksare switched in use in order to execute data recognition by combinationsof a plurality of neural network processing. Further, in the same way aswith the case of the character recognition, it is also possible toincrease the recognition capability by combinations of the conventionalmethods. The final recognition results are transmitted, together withthe original image and the pre-processed image, to a monitoringapparatus 903 as monitoring information and to terminal control loops(not shown) as control reference command. The architecture of the datarecognition system 905 will be described in further detail hereinbelow.

FIGS. 13 to 15 show some architecture examples of the large scale datarecognition systems 905.

In the recognition system, shown in FIG. 13, a learning withoutteaching-data network B01 first executes a rough categoryclassification, and thereafter a learning with teaching-data network B02outputs the final recognition result on the basis of the categoryclassified data, to which the system shown in FIG. 4 is applied.

FIG. 16 shows a processing image of the learning without teaching-datatype network B01. In case of the learning without teaching-data typenetwork, basically each unit receives all the outputs of an image to beprocessed, internally processes the received image data, and learns insuch a way as to form a category of the inputted image data, with theunits for outputting the largest values as its center (the shaded unitsin FIG. 16 imply the central units of the category).

After having learned the sufficient image to be processed, the learningwithout teaching-data type neural network B01 can output the classifiedcategory of the newly presented image data to be processed. In order toassociate these data with symbolic recognition, the learning withteaching-data type neural network B02 is prepared for succeedingprocessing. FIG. 17 shows the processing image of the learning withteaching-data type neural network B02. In FIG. 17, the data outputted bythe respective units of the output layer are compared with the patternclassification teacher data, and the learning is executed so that thedifference between the two can be reduced to zero.

The category classification by the learning without teacher-data typeneural network B02 is a rough and preliminary classification in acertain sense. For executing further definite information compression,an architecture as shown in FIG. 14 is used, in which functional modulesfor extracting features are additionally incorporated. In this case,when a conventional feature extracting apparatus B03 is adoptable, it ispossible to incorporate the conventional apparatus with the system.

FIG. 18 shows a processing image obtained when the feature extractingnetwork B04 is incorporated. In this network, the number of the units ofthe input layer is equal to that of the output layer. Further, thefollowing relationship can be established: The number of units of theinput layer is much greater than the number of units of the secondintermediate layer (the number of units in the central layer, ingeneral). Since the network learns so as to output the same data as theinput image, the network is substantially the same as the ordinarylearning with teaching-data type network. The output pattern of thecentral intermediate layer (the second intermediate layer in FIG. 18)obtained after the learning includes sufficient information forreproducing the input image at the network output layer. Therefore, itis possible to interpret that the output pattern thereof is a signalvector indicative of a feature variable. Accordingly, the networkconstructed by the three layers (the input layer and the first andsecond intermediate layers) of the first half of the network can be usedas the feature extracting network B04 after learning. Further, the finalfeature parameters obtained by a feature parameter select and synthesisunit B05 are given to the learning with teaching-data type neuralnetwork B02 as the final recognition results, together with the otherfeature parameters when the conventional feature extracting methods canbe utilized.

Further, when the obtained feature variable data are redundant, it ispossible to construct the system such that a learning withoutteaching-data type network B01 for executing the category classificationis further incorporated with the system as a functional block,a s shownin FIG. 15.

Further, as a system architecture which is further expanded beyond thoseshown in FIGS. 14 and 15, it is of course possible to assemble a furthergreat number of teacher-absent and learning with teaching-data typeneural networks for executing further detailed and fine functionalclassification and recognition. In this case, it is possible to realizethe recognition system in a practical level with the use of a smallnumber of hardware resources, as far as the above-mentioned architectureof the large scale data classification and recognition system can beadopted.

In the system (as shown in FIG. 10) used as the system for recognizingthe char-bed shape within the recovery boiler furnace, it is possible toconstruct apparently a multifunction large scale neural network by useof a small number of hardware resources and by switching networkinformation required for one network hardware. In this case, it ispossible to realize a single chip system.

FIG. 19 is a block diagram showing an example of the above-mentionedhardware construction.

First, in FIG. 19, a neural network hardware element D01 includes anarithmetic circuit, RAM, ROM, etc. for implementing the basiccomputations as a neural network element. The neural network hardwareelement D01 is connected to a neural network connection weight memoryD02 and a neural network element input/output signal memory D03 throughan internal data bus, in order to transfer connection weight databetween the respective neural network elements and input/output signaldata of the respective neural network elements. The neural networkconnection weight D02 saves the transferred connection weights betweenthe respective neural network elements, and the neural network elementinput/output signal memory D03 saves the transferred input/output signaldata of the respective neural network elements. The data transfer amongthe neural network hardware element D01, the neural network connectionweight memory D02, and the neural network element input/output signalmemory D03 is controlled by an internal data bus controller D07.Further, the data transfer between the neural network elements and theexternal devices is controlled by an external data bus controller D04.Since the internal data bus controller D07 controls the data transferbetween the neural network hardware element D01 and the memories D02 andD03, it is possible to improve the data transfer speed. A programcontroller D05 sets the software and executes the sequence control inaccordance with the software, when one neural network hardware elementD01 is used as a plurality of neural network elements. Further, anaddress generating section D06 generates addresses of the data to bewritten in the neural network hardware element D01 and the memories D02and D03 in accordance with the instruction of the program controllerD05.

The learning processing of this neural network chip can be executed inaccordance with a forward direction propagation algorithm shown in FIG.20 and a reverse direction propagation algorithm shown in FIG. 21, andthese algorithms are executed under control of the program controllerD05 shown in FIG. 19.

First, with reference to the forward direction propagation controlalgorithm shown in FIG. 20, in step E01, control reads all theconnection weights W_(ij) connected to the inputs of the neural networkelement i from the memory D02, where i denotes each neural networkelement realized by software when a single neural network element D01 isused as a plurality of neural network elements in the form of software.In step E01, therefore, the connection weights W_(ij) of a single neuralnetwork element i are read from the memory D02. Successively, in stepE02, Y_(ji) (transmission signals) from the neural network element i tothe neural network element j are read from the memory D03. Further, ithas been confirmed that all the data necessary for the computing ofneural network element i's output are read from the memories D02 and D03to the neural network hardware element D01 in step E03. The neuralnetwork hardware element D01 is operated for computation to obtain thecomputation results in step E04. In step E05, the obtained computationresults are written in the memory D03 as the transmission signals Y_(ik)to be transmitted from the neural network element i to another neuralnetwork element k when considered in the form of software. Aftercomputations have been executed for all the neural network elements i,k, . . . , the output values of the final neural network can beobtained.

After the learning data have been propagated in the forward direction,the error reverse direction propagation is executed by calculating theconnection weights between the respective neural network elements on thebasis of an error between the neural network output value and thecriterion output. Refer to COMPUTROL, No. 24 pp 53 to 60 for furtherdetail of the practical computation algorithm.

Here, with reference to the reverse direction propagation controlalgorithm shown in FIG. 21, first the step F01 includes the algorithmshown in FIG. 20. Therefore, in this step, the output of the neuralnetwork is calculated in accordance with the forward directionpropagation control algorithm on the basis of leaning data and inaccordance with the forward direction propagation. Thereafter, thereverse direction propagation learning starts. First, in step F02, thepropagation error to the preceding intermediate layer between two outputlayers and the updated connection weights are calculated, on the basisof the error between the output signal and the criterion output of theneutral network and the connection weights to the output layers (storedin the memory D02) and in accordance with the error reverse propagationalgorithm. Further, the connection weights are updated in the order fromthe output layer to the input layer. That is, in step F03, theconnection weight W_(ij) from the neural network element j and theneural network element i to which the output of the neural network i isinputted is read from the memory D02. Further, in step F04, the outputvalues of the learning data of the neural network element i are readfrom the memory D03 to the neural network hardware element D01. Further,in step F05, the propagation errors calculated in order from the outputlayer are read from the memory D03. Thereafter, in step F06, thesucceeding error propagating in the intermediate layer and the updatedconnection weights W_(ij) are calculated in accordance with the errorreverse direction propagation algorithm. Further, in step F07, thecalculated propagation error and the connection weights are stored inthe memory D03 for data update. Thereafter, in step F08, control checkswhether all the calculations of all the neural network elements havebeen completed from the software standpoint. The above-mentioned stepsF03 to F07 are repeated until the result of `YES` can be obtained.

As described above, in case of a single-chip neural network system,since a large scale neural network system can be constructed withoutincreasing the number of neural network hardware elements and since thespace required for the neural network system and the number of theconnection wires can be reduced, the reliability of the neural networksystem can be improved. Further, since the data can be transferredbetween the neural network hardware elements and the neural networkstoring RAM through the internal bus constructed within the same chip,the data can be transferred at high speed, so that it is possible toshorten the calculation time required for the neural network system. Inaddition, since these elements can be constructed on the same chip, itis possible to improve the reliability of the hardware elements.

As described above, in the information recognition system according tothe present invention, it is possible to realize the informationrecognition system which can acquire a great number of recognition rulesautomatically through the processing of learning data (these have beennot so far realized). Further, the number of iterative calculationsrequired for acquiring rules can be reduced down to a practical number.In addition, since each neural network unit can be composed of aboutseveral tens of neurons in general, the number of the network connectioncircuits can be reduced down to a realizable number. Further, since thenetwork connection circuits between the units can be composed of theinput and output signal lines between the respective units, the numberof the circuits is realizable. Therefore, it is possible to solve theproblem that the number of the connection circuits increasesdrastically, which has been so far involved in the conventional largescale neural network system.

Further, in the case where the category classification by the clusteringprocessing is executed at a plurality of stages, as far as the number oflearning data belonging to the finally classified categories issufficiently reduced, it is possible to roughly secure the convergenceof the learning by use of the category classification recognition systemprovided with a hierarchical neural network architecture. Therefore, itis possible to construct the recognition system under the practicalengineering load and further to realize appropriate high-speedrecognition and classification for a large scale data group.

Further, based on the conception such that a limited number of networkhardware resources can be used as a great number of networks byswitching data inputted to the network hardware resources, theprocessing executed by a great number of neural networks can be realizedby a minimum possible number of practical processors, thus allowing thelarge scale data recognition to be applicable to various fields inpractice. Further, in case of a single-chip neural network system, sincea large scale neural network system can be realized without increasingthe number of neural network hardware elements, the space required forthe neural network system and the number of the connection wires can bereduced. Accordingly, the reliability of the neural network system canbe improved. Further, since the data can be transferred between theneural network hardware elements and the neural network storing RAMthrough the internal bus constructed within the same chip, the data canbe transferred at high speed, so that it is possible to shorten thecalculation time required for the neural network system. In addition,since these elements can be constructed on the same chip, it is possibleto improve the reliability of the hardware elements.

FIGS. 22 to 24 show a first embodiment of a man-machine interface system(a control system) using the large scale information recognition systemaccording to the present invention.

In FIG. 22, a microcomputer body 221 includes a CPU, a ROM for storingoperation programs and data, a RAM, an image memory, interfaces, etc.,in addition to an image signal acquiring board (described later), animage processing board, a neural network board, a control signalcalculation board, etc. Further, a TV camera 222, a display 223, a keyboard 224, a mouse 225, etc. are connected to the microcomputer 221. Onlayout of this system, the display 223 is mounted on the computer 221,and the TV camera 222 is mounted at the central portion on the uppersurface of the display 223.

In the layout as described above, the camera 222 takes an image of anoperator (his whole face, in particular), and transmits the imagesignals to the computer 221. The computer 221 converts the transmittedanalog image signals to digital image signals, and recognizes thefeeling of the operator on the basis of the converted digital data.Further, on the basis of the recognized operator's feeling, the computer221 extracts an operating instruction most appropriate to the currentfeeling of the operator from a plurality of operation instructionspreviously stored in the ROM, and displays the extracted instruction onthe display 223. With reference to the displayed instruction, theoperator can proceed-with his procedure by use of the keyboard 224 andthe mouse 225.

FIG. 23 is a block diagram showing the man-machine interface system asdescribed above, and FIG. 24 is a flowchart showing an example of theoperation thereof.

The system shown in FIG. 23 comprises the TV camera 222, the display223, the image signal acquiring board 236, the image processing board237, the neural network board 238, and the control arithmetic board 239.

The image signals taken by the TV camera 222 are A/D converted by theimage signal acquiring board 236, and then inputted to the imageprocessing board 237. The image processing board 237 stores the inputteddata once in the memory and then extracts only necessary image data onthe basis of other data in the memory. In FIG. 24, an illustration OP1shows an uneasy feeling of the operator which can be understood on thebasis of the shapes of his eyes, eyebrows and mouth. Therefore, theimage processing board 237 first executes the edge detection to extractthe shapes of the eyes, eyebrows and mouth (which provide key pointsrelated to the operator's facial expression).

The image data extracted by the image processing board 237 aretransmitted to the neural network board 238. Since the neural networkconstructed as shown in FIGS. 1A and 1B has already learned how torecognize the operator's facial expression and feeling on the basis ofthe given image data, the neural network board 238 recognizes theoperator's facial expression on the basis of the image data transmittedby the image processing board 237, and outputs the recognized results.When the operator's eyes, eyebrows and mouth express an uneasy facialexpression as show by OP1 in FIG. 24, the expression is discriminatedand "impatient" (YES) is determined in OP2 in FIG. 24.

In this case, the control signal calculation board 239 extracts anappropriate operating instruction from the memory in accordance with theoutput of the neural network board 238 as shown by OP3 in FIG. 24 anddisplays the extracted instruction. In more detail, if YES in OP2, aneasy instruction for non-skilled operators is selected, and if NO inOP2, a simple instruction for skilled operators is selected in OP3. Theselected instruction is displayed on the display 223 in OP4.

As described above, since the above-mentioned operation is executed atall times, it is possible to select an appropriate operating instructionaccording to the change in the facial expression of the operator.

Further, in the above-mentioned system, although the operator's eye,eyebrows and mouth have been detected, it is also possible to detect theperspiration on the operator's face in order to effectively discriminatethe degree of the irritation of the operator. Further, when the wrinklesare detected, it is possible to effectively detect the feeling ofelderly men usually not skilled in the system of this sort. In addition,it is also possible to use a speech output apparatus to give a timelyadvice to the non-skilled operator.

Further, in the above-mentioned system, although the image processingboard 237 executes the edge detections to extract the facial expressionfactors such as the eyes, eyebrows and mouth, it is also preferable touse the neural network to extract the facial expression factors.

Here, the case is taken into account where neural network as shown inFIG. 1A extracts the operator's eyes, eyebrows and mouth on the basis ofthe whole image of an operator face, by way of example. The roles areallocated to the respective units NN, for instance as follows: theneural network unit NN₁₁ extracts eyes from the whole face picture; NN₂₁corrects the positions and sizes of the extracted eyes in accordancewith the recognition rules; NN₃₁ detects a tense eye pattern; NN,₂extracts eyebrows from the whole face picture; NN₂₂ corrects thepositions and sizes of the extracted eyebrows in accordance with therecognition rules; NN₃₂ detects a tense eyebrow pattern; NN₁₃ extracts amouth from the whole face picture; NN₂₃ corrects the positions and sizesof the extracted mouth in accordance with the recognition rules; NN₃₃detects a tense mouth pattern, respectively.

In this case, the image data representative of the whole face are givenas the learning input image data for the unit NN₁₁, and the dataindicative of the eyes are given as the teaching data. Further, everypossible patterns are given and learned until the image signal outputcan extract the eyes. By the above-mentioned learning, the unit NN₁₁connotes the overall feature detecting capability including the eyeshape and the positional relationship between the eye shape and theother parts in the face, so that it is possible to specify the eyes fromthe whole face image, in the same way as with the case where a man canjudge the presence of the eyes by seeing the man face.

As the learning input image data of the unit NN₂₁, various patternsindicative of eyes of various sizes existing at various positions (i.e.,various patterns expected to be outputted by the unit NN₁₁, when thesystem is used in practice) are given. As the teacher data, the patternscorresponding thereto and further corrected in position and size aregiven. The learning is repeated on the basis of the given learning andteacher data. Accordingly, the unit NN₂₁ can soon output an eye patternobtained by correcting the eyes of various sizes existing at variouspositions to the eyes of a constant size existing constant positions.

The various eye patterns of various shapes of a determined eye sizeexisting predetermined positions (i.e., various patterns predicted to beoutputted by the unit NN₂₁) are given as the learning data of the unitNN₃₁, and the data indicative of whether the corresponding pattern is atense pattern ((1) Positive) or not ((0) Negative) are given as theteacher data. The learning is repeated until the answer of (1) or (0)can be obtained at a constant rate (i.e., until a correct answer can beobtained except the case where it is difficult to discriminate whetherthe eyes indicate tense feeling or not when seen by a man).

The same as above is applied to the case of the eyebrows and mouth. Insummary, the image data (learning input image data) indicative of thewhole face and the image data (teacher data) indicative of the eyebrowsare given to the unit NN₁₂. Further, the eyebrow patterns (learninginput image data) of various sizes existing at various positions and theeyebrow patterns (teacher data) of a predetermined size existing atpredetermined positions are given to the unit NN₂₂. Further, the variouseyebrow patterns (learning input image data) of various shapes of apredetermined size existing at predetermined positions and the dataindicative of whether the corresponding patterns is a tense pattern ((1)Positive) or not ((0) Negative) are given to the unit NN₃₂ forrespective learning.

In the same way, the image data (learning input image data)representative of the whole face and the image data (teacher data)indicative of the mouth are given to the unit NN₁₃. Further, the mouthpatterns (learning input image data) of various sizes existing atvarious positions and the mouth patterns (teacher data) of apredetermined size existing at a predetermined position are given to theunit NN₂₃. Further, the various mouth patterns (learning image data) ofvarious shapes of a predetermined size existing at a predeterminedposition and the data indicative of whether the corresponding patternsis a tense pattern ((1) Positive) or not ((0) Negative) are given to theunit NN₃₃ for respective learning.

Further, after the respective learning have been converged, therespective change-over switches SW are set to the b side. In thepractical use, it is possible to obtain a final decision owing to thelinked functions of the respective units.

Here, it is also possible to consider that the system can be constructedby the neural network by allowing a large scale neural network to learndata, without depending upon the combination with a number of theabove-mentioned small units. In this case, although the system itselfcan be constructed, since the factors to be learned becomes huge, thelearning operation is complicated, so that it takes a long time toconverge the learning operation and thereby the scale thereof islimited.

In contrast with this, when the small units are combined with eachother, since the learning operation can be converged for each unit, itis possible to execute the learning effectively in short time. Inaddition, when all the units are learned in parallel to each other, evenif the scale increases, the learning time is not increased, so that thesystem is not subjected to the limitation of the scale in this sense.

Therefore, in the above-mentioned embodiment, although the neuralnetwork composed of nine units as shown in FIG. 1A has been explained,it is of course possible to construct a further larger scale neuralnetwork, without being limited only to the system shown in FIG. 1A.

Further, there exists another problem with respect to the person skilledin computer operation to some extent. The problem is related to theoperation of inputting data to the machine through the display picture,in particular with the use of a pointing device. As the pointingdevices, a touch pen, mouse, etc. are so far known. In these pointingdevices, a pointer is shifted by the operator's hand, so that arelatively large operation is required for the operator whenever thepointer is required to be moved. For instance, when a cursor on adisplay picture is moved with the use of a mouse, the operator mustfirst takes a mouse with his hand, moves the mouse on a predeterminedplace (a mouse pad, a desk, etc.), and then clicks the button on whenthe cursor is located. In these operations, since some operation stepsare required, there exists the case where the cursor cannot be placed toany desired positions along a considered locus, thus causing a viciouscycle of irritation and erroneous operation, in spite of the fact that aquick operation is required for the operator.

In the above-mentioned embodiment, although the neural network isadopted, another object is to extract the visual information of theoperator on the basis of the image signals and further to control thepointer device on the basis of the extracted visual data.

FIGS. 25 to 27 show a second embodiment of the man-machine interfacesystem according to the present invention.

FIG. 25 is a block diagram showing the hardware construction thereof.The man-machine interface system shown in FIG. 25 can be divided roughlyinto a system for processing the image recognition and a system forprocessing the display. The image recognition processing systemcomprises a CPU 251, a bus 252, a ROM 253, a RAM 254, two frame memories(RAM) 255 and 256, a TV camera 257, an A/D converter 258, a switch 259,and a parallel interface 261. The display processing system is composedof a CPU 261, a display controller 262 and a display 263.

The CPU 251 controls various operation in accordance with programs anddata stored in the ROM 253. The RAM 254 is used for various purposessuch as buffers, registers for displaying statuses, etc. in the controloperation. Therefore, the current position of the pointer is stored inthe RAM 254.

The TV camera 257 is disposed on the upper portion of the display 263,in the same way as with the case of the first embodiment of theman-machine interface system, to take an image of the upper half of theoperator so that at least the whole operator's face can be image sensed.The A/D convertor 258 converts the analog image signals transmitted fromthe TV camera 257 to digital image signals of 4 bits (16 gradations) perpixel, for instance in accordance with the control of the CPU 251. Thedigital image signals for one frame are stored in the RAM 255. The CPU251 binarizes (described later) the data stored in the RAM 255 andstored them again in the RAM 256 so as to be used as the pointer controldata.

The switch 259 is composed of predetermined keys of the keyboard, thedigitizer, etc. and a click button of the mouse, which are used totransmit an operator's response to a displayed message or to select amenu designated by the pointer.

The CPU 261 of the display processing system controls the communicationwith the CPU 251 through the parallel interface 261 or the displaycontroller 262. In response to the display control instructions from theCPU 251, CPU 261 commands the display controller 262 to executeprocessing in accordance with the instructions. In accordance withinstructions of the CPU 261, the display controller 262 controls thedisplay 263. For instance, in the case where the instruction of the CPU261 is a pointer moving instruction, the display controller 262 movesthe pointer to a designated position on the picture of the display 263.

FIG. 26 shows a pointer control program stored in the ROM 253, that is,a flowchart of the procedure of the CPU 251.

When power is turned on (in step S101), the CPU (referred to as control,hereinafter) 251 executes initialization to clear the internalregisters, counters, RAMs 254 to 256 (in step S102).

Further, control displays the pointer on a reference position (X1, Y1)on the display 126 (in step S103) and displays a message of "Depressswitch 259 by seeing the pointer" on the display 263. When the switch259 is depressed, control detects the eye positions and stored thecoordinates (x1, y1) of the detected eye positions in the RAM 254 (instep S104).

Control deletes the pointer at the position (X1, Y1) on the display 263,and displays another different pointer at a position (X2, Y2), andfurther displays a message of "Depress switch 259 by seeing the pointer"on the display 263 (in step S105). Here, the position (X2, Y2) isselected so that the following matrix becomes a full rank: ##EQU1##

Further, control detects the eye positions again (in step S106), andstored the coordinates (x2, y2) in the RAM 254, and deletes the pointerand the message on the display 263.

Control obtains 2×2 matrix A on the basis of the two points (X1, Y1) and(X2, Y2) on the display 263 and the two points (x1, y1) and (x2, y2)corresponding thereto on the image data in accordance with the followingformula: ##EQU2##

The calculated matrix is stored in the RAM 254 (in step S107).

This matrix A is a coordinate conversion matrix for determining thepointer position on the display on the basis of the eye positions of theimage data taken by the TV camera 257.

After the above-mentioned conversion matrix A has been obtained, controldetects the eye positions on the basis of the new image data taken bythe TV camera 257, and stores the coordinates (x, y) in the RAM 254 instep S108. Further, control calculates the pointer position (X, Y) to bedisplayed on the display 263 in accordance with the following formula instep S109: ##EQU3##

The calculated pointer position signals are outputted to the parallelinterface 261 in step S110, and control returns to the eye positiondetection in step S108.

Upon reception of the pointer position signals from the parallelinterface 261, control commands the display controller 262 to output thepointer position.

When the switch 259 at hand is depressed, in response to this switch-onsignal, control 251 selects the current pointer position signals.

FIG. 27 shows the detailed eye position detecting processing executed insteps S104, S106 and S108 of the flowchart shown in FIG. 26.

With reference to FIG. 27, the operation of the microprocessor (CPU) 25shown in FIG. 25 will be described hereinbelow. CPU (control) firstwrites image data for one frame obtained by the TV camera 257 in the RAM255 (in step S201), binarizes these image data (in step S202), andstores the binary data indicative of the presence or absence of imagedata of one-bit for each pixel (in step S203).

Then, the central positions of the pupils of the eyes on the basis ofthese binary data are detected (in step S204).

Upon failure of the central position detection, detection-check counters(not shown) are incremented (in step S206), and further checks whetherthe counter value reaches some value, for example, 15, which is easy tobe expressed by a 4-bit counter (in step S207). If does not yet reach15, control decrements a binary reference value (in step S209) andre-executes the binarization processing (in steps S202 and 203) and theposition detection (in step S204) on the basis of the changed referencevalue. When the position detection fails 16 times, the image data arejudged not to be appropriate for detection processing and thedetection-check counters are cleared (in step S208), returning to thepointer display. The causes of failure may be the absence of eyes'imageon the data, poor contrast of the image, etc. In this case, the imagedata are sensed again to detect again the eye position detection.

Upon success of the pupil position detection, the detection-checkcounters are cleared (in step S205), and proceeds to any of steps S105,S107 and S109 shown in FIG. 26.

As described above, in the first embodiment of the man-machine interfacesystem according to the present invention, the facial expression of theoperator is always monitored; the feeling of the operator is guessed inorder to check the skillfulness of the operator in system operation onthe basis of the image signals; and the operating instruction display isselectively switched. Therefore, it is possible to select an appropriatedisplay instruction according to the operators, with the result that itis possible to provide an appropriate instruction to the operator at alltimes and to improve the operating efficiency of the operator, beingdifferent from the conventional predetermined or fixed operatinginstruction.

Further, in the conventional system, an adviser is always required forthe automatic cash dispenser, for instance in order to assist old oryoung operators who cannot handle the cash dispenser on the basis ofonly fixed operating instruction. In case of the system according to thepresent invention, it is possible to eliminate such an adviser andthereby to automatize the attendance on customers without trouble.

Further, in the second embodiment of the man-machine interface systemaccording to the present invention, the operator's eye motion isdetected on the basis of image signals, and the pointer is movedaccording to the detected eye motion so that the pointer can be moved onthe display picture in response to the movements of the operator's eyesor face. Accordingly, it is possible to increase the operation speed andto decrease the operation difficulty, without moving his hand so much.

A one-loop controller using the information recognition system accordingto the present invention will be described using the attached drawings.FIG. 28 is a block diagram showing an embodiment of a process controllerof the present invention. The process controller comprises an inputinterface section 281, an image recognition section 282, a controlsection 283, and an output interface section 284. The input interfacesection 281 acquires video signals inputted by a TV camera or CCD camera(both not shown), converts the acquired data into image pattern dataprocessed by the computer, and stores the converted image data in amemory of the interface CPU. The image recognition section 282 outputsrecognition results of the image information transmitted by the inputinterface section 281 and in accordance with a recognition algorithm.

FIG. 29 shows an example of the image recognition section 282 composedof an image processing section 291 and a recognition section 292. Theimage processing section 291 extracts the feature variables of thevarious feature parameters Xn of the image by detecting image edges orin accordance with an algorithm such as FFT (first Fourier transform)analysis. The recognition section 292 (which is a multilayer neuralnetwork as shown in FIG. 1A) using the extracted various featureparameter values classifies the parameter values by pattern matching,and outputs classification matching scores.

FIG. 30 is a block diagram showing a multilayer neural network in whichthe number of the inputted feature parameters is n,, and the number ofthe outputted classifications is n_(M). Further, the weight coefficientsat the respective nodes of the neural network is learned by giving theclassifications of the various feature parameters as teacher data usingthe known method such that back propagation method, Vogl method, etc.

In FIG. 28, the control section 283 executes algorithms to obtainoperation parameter set values of a process to be controlled and controlcommands such as operation commands for actuators within a plant,according to pattern recognition results of the image recognitionsection 282.

An example of algorithm of the control section 283 will be explainedwith reference to FIG. 31.

In response to the classification adaptation which is the recognitionresults of the neural network of the recognition section 292, thecontrol section 283 decides the control commands on the basis of fuzzyinference.

For instance, in the case where an air pressure set value and an airflow rate set value are changed in a plant boiler by classificationresults of the image patterns, the rules used for the fuzzy inference isas follows:

If the pattern is 1, the air pressure set value is increased and the airflow rate set value is decreased;

If the pattern is 2, the air pressure set value is increased and the airflow rate set value is kept as it is;

:

:

:

If the pattern is n_(M), the air pressure set value is decreased and theair flow rate set value is increased.

When the outputs of the control section 283 are the respective setvalues, the inferred results are added to the preceding set values andthen outputted. When a subordinate controller is provided with functionsof calculating the set values, it is also possible to output the changerates of the set values directly.

In the fuzzy inference of the present invention, the recognition resultsof the neural network are used as the adaptations as they are, withoutcalculating the adaptations in the condition section as with the case ofthe ordinary fuzzy inference. Further, in the conclusion section, theconventional method is used such as MAX gravity center method, addedgravity center method, etc., for instance. Further, FIG. 31 is anexample of the conclusion membership functions of the above-mentionedrules adopted for the fuzzy interference.

The output interface section 284 shown in FIG. 28 outputs thecalculation results of the control section 283 to process actuators (notshown) or respective controllers of subordinate distribution-typecontrol systems.

FIG. 32 is a block diagram showing another one-loop controller accordingto the present invention. In this embodiment, in addition to theone-loop controller shown in FIGS. 28 and 29, there is provided acontrol parameter adjusting section 326 (having an interface section forthe external signals inputted by the operator, for instance) foradjusting the parameters of the control section 283, from the outputsignals of the recognition section 292 and the control section 283.

The control parameter adjusting section 326 is described in furtherdetail hereinbelow. As shown in FIG. 33, the control parameter adjustingsection 326 is provided with a saving function 326a for storing therecognition results of the neural network and the control commands bythe recognition results as time-series data, a recognition resultdisplay function 326b for displaying the time trend of the neuralnetwork recognition results saved by the saving function 326a, a controlcommand display function 326c for displaying the time trend of thecontrol commands saved by the saving function 326a, a rule correctingfunction 326d for correcting the rules used for the fuzzy inference, anda membership function correcting function 326e for correcting themembership functions used for the fuzzy inference.

The respective functions of the control parameter adjusting section 326will be described in detail hereinbelow with the use of the operationdisplay pictures on the computer display. FIG. 34 is an example of theoperation display picture, in which the man-machine interface related tothe saving function 236a can be achieved by a start save icon 342, anend save icon 343, and a result display screen 344. On the resultdisplay screen 344, the recognition results and the control commands aremonitored and displayed as time trend graph. When the operatordesignates the save start icon 342 with a pointing device such as amouse, a touch pen, etc. while seeing the monitor picture, thetime-series data of the recognition results and the control commands arestarted to be saved. Further, when the operator designates the save endicon 343, the saving ends. The recognition result display function 326bcan be achieved by a recognition result display screen 344 and arecognition result display icon 345. When the display icon 345 isdesignated, the time trend graph of the recognition results of therecognition section saved as the adjusting data by the saving function326a is displayed on the result display screen 344. The man-machineinterface related to the control command display function 326c can beachieved by the result display screen 344 and the control commanddisplay icon 346. When the display icon 346 is designated, the timetrend graph of the control commands saved as the adjusting data by thesaving function 326a is displayed on the result display screen 344. Theman-machine interface related to the rule correcting function 326d canbe achieved by a rule correct start icon 347, a rule correct end icon348 and a correct display screen 349. When the rule correct start icon347 is designated, a rule correcting picture as shown in FIG. 35 isdisplayed on the correct display screen 349. The rules are corrected bychanging (increasing or decreasing) the fuzzy label. For the change, oneof the fuzzy labels already registered is selected or a new fuzzy numberis entered. When the rule correct end icon 348 is designated, the rulecorrecting function 326d ends after having confirmed the presence orabsence of the data save. The membership function correcting function326e can be achieved by a membership function correct start icon 350, amembership function correct end icon 351, and the correct display screen349. When the membership function correct start icon 350 is designated,a membership function correcting picture as shown in FIG. 36 isdisplayed on the correct display screen 349. The membership function iscorrected by changing the shape parameters. Further, when the membershipfunction correct end icon 351 is designated, the membership functioncorrecting function 326e ends after having confirmed the presence orabsence of the data save.

FIG. 37 shows another construction of the one-loop controller. In thisembodiment, in addition to the one-loop controller shown in FIGS. 28 and29, there is provided a weight coefficient learning section 377 (havingan interface section for the external signals inputted by the operator,for instance) for learning the weight coefficients of the neural networkof the recognition section 292, from the output signals of the inputinterface section 281 and the recognition section 292.

As shown in FIG. 38, the weight coefficient learning section 377comprises an image data monitoring function 377a for monitoring theimage data, an image data saving function 377b for saving the imagedata, an image data re-display function 377c for displaying the imagedata again, a learning data selecting function 377d for selecting anddeleting the learning image data, a learning data input function 377efor inputting the teacher data for learning, a learning conditionsetting function 377f for designating the learning start and end and thenumber of learning times of the weight coefficients of the neuralnetwork, an error trend display function 377g for displaying the errortrend between the neural network outputs and the teacher data, and aconnection weight display function 377h for displaying the connectionweight of the neural network.

The respective functions of the connection weight learning section 377will be described in detail hereinbelow with reference to operationdisplay pictures. FIG. 39 shows an example of the operation displaypicture. In the drawing, the man-machine interface related to the imagedata monitoring function 377a can be achieved by a display screen 310, amonitor start icon 320, and a monitor end icon 330. When the monitorstart icon 320 is designated, the image data taken by the TV camera aredisplayed on the display screen 310, and when the monitor end icon 330is designated, the image data monitoring ends. The man-machine interfacerelated to the image data saving function 377b can be achieved by a savestart icon 340 and a save end icon 350. When the save start icon 340 isdesignated, the image data being monitored are started to be saved, andwhen the save end icon 350 is designated, the image data saving ends.The man-machine interface related to the image data re-display function377c can be achieved by the display screen 310, an re-display icon 360,and a re-display range designating bar 370. When the redisplay icon 360is designated, the saved image data are displayed on the display screen310. When a re-display time range is designated by the re-display rangedesignating bar 370 and the re-display icon 360 is designated, the savedimage data are displayed on the display screen 310 in time seriesfashion within the designated time range. When the re-display icon 360is designated again, the re-display ends. The man-machine interfacerelated to the learning date selecting function 377d can be achieved bya learning data select icon 380, a data number display 390, a learningdata display icon 400, and a learning data delete icon 410. When thelearning data select icon 380 is designated during the data monitor orre-display, a data number is automatically attached to the image datanow being displayed, and then the data are registered. The attached datanumber is displayed on the data number display 390. When the learningdata display icon 400 is designated, the registered learning data aredisplayed on the display screen 310. If unnecessary data exists, whenthe unnecessary data is designated and further the learning datadeleting icon 410 is designated, it is possible to delete the learningdata. In this case, the data numbers are corrected automatically. Theman-machine interface related to the teacher data input function 377fcan be achieved by a teacher data input icon 420 and the display screen310. When the teacher data input icon 420 is designated, teacher datainput windows for the registered learning image data and theclassification adaptation are displayed on the display screen 310 asshown in FIG. 40, so that teacher data corresponding to the learningdata are inputted. The man-machine interface related to the learningcondition setting functions 377f can be achieved by a learning numberinput display 430, a learning start icon 440, and a learning end icon450. When the number of learning times is entered to the learning numberinput display 430 and further the learning start icon 440 is designated,the weight coefficient learning section 377 executes the connectionweight learning from the registered learning data and teacher data usingthe back propagation method or the Vogl method, etc. When the number oflearning times reaches the designated number or when the learning endicon 450 is designated, the weight coefficient learning ends. Theman-machine interface related to the error trend display function 377gcan be achieved by an error trend display icon 460 and the displayscreen 310. When the error trend display icon 460 is designated, theerror trend between the neural network outputs already learned or nowbeing learned and the teacher data are displayed on the display-screen310. The man-machine interface related to the weight coefficient displayfunction 377h can be achieved by a weight coefficient display icon 470,a layer number designate display 480, and the display screen 310. Whenweight coefficients are entered to the layer number designate display480 and further the weight coefficient icon 470 is designated, theweight coefficients after learning end are displayed on the displaypicture 310.

Another embodiment of the one-loop controller will be describedhereinbelow with reference to FIG. 41. In the drawing, an inputinterface section 281 and an image recognition section 282 are the sameas with the case of the controller shown in FIG. 37. A control section413 of the one-loop controller is composed of a controller 413a foractually controlling a plant and a simulator section 413b for simulatingthe controller 413a. Further, as shown in FIG. 42, the simulator section413b is composed of an image data forming section 421, a controlsimulation section 422, a control object model 423, and a controlconstant adjusting section 424. The simulator section 413b simulates thecontroller 413a according to the following procedure to evaluate thecontrol performance.

First, image data for simulation are formed. As shown in FIG. 43, on animage data forming menu screen 500, there are arranged an image displaywindow 510, a monitor icon 520, an image save start icon 530, an imagesave end icon 540, a re-display icon 550, a re-display range designatebar 560, a data cut-off icon 570, a data display icon 580, a data deleteicon 590, and a data number input display area 600. An image now beingmonitored is displayed in the image display window 510. When the datacut-off icon 570 is designated by a mouse for instance, the image frameat the current time point is numbered automatically as an image datafile and then registered. Further, when the data save start/end icons530 and 540 are designated, the image can be saved for any given longtime. When the re-display time range of the saved image is set by there-display range designating bar 560 and further the re-display icon 550is designated, the saved image frames are displayed on the image displaywindow 510 in time series fashion within a designated time range.Therefore, it is possible to cut off the image data by seeing theredisplayed image and save the cut-off data as another file. When thedata display icon 580 is designated, the image window is divided intosome display regions at need to display the image data so far formed asmultiple pictures. In this case, when any overlapped data exist, theoverlapped data display region is designated and then the delete icon590 is designated to delete it. When the monitor icon 520 is designatedafter the saved image is redisplayed or after the image data aredisplayed, it is possible to display the image now being monitored againin the image display window.

As described above, since necessary image data required for thesimulation can be formed, it is possible to obtain sufficient data forthe adjustment.

After the image data have been formed, the control performance isevaluated by the controller simulation. As shown in FIG. 44, on thecontrol simulation menu screen 700, there are arranged an image datadisplay window 710, an image data display mode switching icon 720, animage data selecting icon 730, a simulation executing icon 740, anoutput result display window 750, and an output result display modeswitching icon 760. The image data display switching icon 720 switches amode for displaying the image data one by one to another mode fordisplaying a plurality of registered image data (by dividing the imagedata display window 710 into a plurality of windows at need) or viceversa. In the case of the mode for displaying data one by one, since thesucceeding candidate image data can be displayed whenever the imageselecting icon 730 is designated, the image data required to be used forthe simulation can be selected. In the case of the mode for displaying aplurality of image data, the displayed image data required to be usedfor the simulation can be designated by selection. Once the data used asthe input image for simulation are selected, the selected image data aredisplayed on the image data display window 710. When the simulationexecute icon 740 is designated, the output results of the respectiveoutput terminals (obtained when the selected image data are inputted)are displayed on the output result display window 750. Here, it ispossible to switch the display format from a numerical type to agraphical type or vice versa by designating the output result displaymode switching icon 760. When the number of the output terminals islarge so that the output results cannot be displayed on the outputresult display window 750 simultaneously, it is possible to see theoutput results by scrolling the display contents with the use of ascroll bar 770.

Accordingly, since it is possible to easily evaluate the controlperformance on various input images and further compare the controlperformance on the same input image while adjusting the control constantby the control constant adjusting section 424 shown in FIG. 42, thecontrol section 413 can be adjusted before installed in the one-loopcontroller.

Another embodiment of the one-loop controller will be describedhereinbelow with reference to FIGS. 45A and 45B. In FIG. 45A, aplurality of the one-loop controller are accommodated in a rack (notshown) so as to be drawn. On the left or right side surface of acontroller body 451, a monitor screen section 452 is provided. Themonitor screen section 452 is of a display of panel type. As shown inFIG. 45B, the screen section 452 is drawn frontward from a rack andfurther opened in the right or left direction so as to be arranged inparallel to the front surface of the controller body 451. On the surfaceof the monitor screen section 452, a monitor screen 454 is assembled todisplay operation parameters such as target values (set value),manipulated variables, controlled variables, etc. in time-seriesfashion. By seeing the operation parameters, the operator can performnecessary operation by use of an input section arranged on a controllerbody front surface 453 and icons (not shown) displayed on the monitorscreen 454.

Further, it is also possible to mount a plurality of small monitorscreen sections 454 on one surface of the monitor screen section 452 orto assemble a plurality of monitor screens 454 on both the surfaces ofthe monitor screen section 452. The monitor screen 454 can beaccommodated within the rack where unnecessary, therefor, there existssuch an advantage that the monitor screen 452 can be prevented fromscratches or dirt.

Further, as shown in FIG. 46, it is also possible to accommodate anumber of controllers 451a to 451h within an accommodating rack. In thiscase, since only a screen section 452d having a necessary monitor screen454d is drawn open as occasion demands, it is possible to arrange thecontrollers without increasing the installation space.

FIG. 47 is another example of the controller provided with a monitorscreen section of slide type on the side surface thereof. On the rightside of a monitor screen 471, a data input panel 472 is arranged.Further, under the monitor screen 471, picture operating keys 473 fordesignating a camera direction, zooming magnification, etc. arearranged. Further, under the data input panel 472, a memory card 474 canbe assembled. The monitor screen 471, the data input panel 472, thepicture operating keys 473 are all mounted on a slide panel 475 whichcan be drawn in the frontward direction so as to be opened in parallelto the front surface of the controller main body. On a narrow frontsurface panel, there are arranged a switch panel 476 for designatingvarious control modes (M: manual mode; A: automatic mode; C: cascademode; and V: visual mode) and a display panel 477 for displayingoperating parameters (PV: control variables; SV: target (set) values;MV: manipulated variables), an external output terminal section 478 towhich a measuring instrument such as a plotter recording apparatus canbe connected, and an earphone terminal 479 for listening sound (e.g.,combustion sound) generated by an object to be controlled.

FIG. 48 shows another example of the picture of the image data displayapparatus according to the present invention, and FIG. 49 shows ahardware construction of the present invention. In FIG. 49, the systemcomprises a visual feedback controller (VFC) (one-loop controller) 501,a local area network (LAN) 502, a TV camera 503, a computer 504, amemory device 505, a CRT display terminal apparatus 506, a mouse 507, awrite pen 508, a touch sensor panel 509 and a keyboard 510, and operatesin the same way as in the ordinary computer.

On the display picture of the image display terminal apparatus 506 ofthe present embodiment, as shown in FIG. 48, a plurality of image databelonging to the same category are displayed simultaneously. Further,various icons for activating various functions for picture operation arearranged on the right side and lower side of the picture. When theseicons are designated by use of the pointing devices such as the mouse507, the write pen 508, the touch sensor panel 509, etc., it is possibleto activate the functions of the designated icons to classify the imagedata effectively.

By use of the function icons 481 to 484 arranged under the-picture shownin FIG. 48, it is possible to activate the functions for shifting thedisplayed picture to the succeeding page, the preceding page, the headpage and the final page, so that the huge image data can be inspectedefficiently. Further, the retrieve function icon 485 activates thefunction of retrieving image data using a numerically represented imagedata feature or a number of the image data.

In FIG. 48, on the right side of the picture, an icon 491 for "Opencategory" is prepared to display the image data of desired category.Further, an icon 492 for "Close category" is prepared to close the fileof the category now being displayed. Further, by use of an icon 493 for"Change category attribute", it is possible to activate a subroutine forchanging the image data features determined by the category. Further, itis also possible to change or correct the evaluation criteria used forthe image data classification by use of this icon 493. Further, by useof an icon 491 of "Form new category", a new category can be formed. Inaddition, an icon 495 of "Option" is prepared as a menu to activatevarious functions. As the optional functions, there are a function forenlarging only the display of designated image data, a function ofrearranging the image data of the same category on the basis of thedesignated feature, a function of moving the designated image data toany given position, a function of converting the designated image datato any given designated category, a function of designating the imagedata display size, a function of designating the image data belonging todifferent categories and displaying the image data on a referencepicture, a function of displaying any given image data or the referenceimage data over any given other image data each other, a function ofdisplaying only image data of designated feature in different color, afunction of determining the order of the image data used for learning, afunction of rearranging the image data belonging to the same category atrandom, etc. Further, when the image data are newly registered, aregistered image data picture is displayed and then the feature of theimage data is memorized as a numerical value according to the evaluationcriterion.

As described above, it is possible to classify the image dataeffectively, by displaying the image data belonging to the same categorysimultaneously so as to be compared with each other in shape.

FIG. 50 shows an example of the monitor picture, in which the shape ofchar bed (ash) of a recovery boiler in a paper plant is monitored by aTV camera of a visual feedback controller for controlling fuel pile andcombustion conditions. Further, in FIG. 50, the contours 50a, 50b,50cand 50d are the char bed shapes required to be recognized. Further,in FIG. 50, numerals 50α and 50β denote dust attached onto the cameralens (that is noise in image information).

When the monitor picture is taken by the neural network and thendisplayed on a picture, the monitor picture shown in FIG. 50 isdisplayed as shown in FIG. 51. In FIG. 51, a pointer 51b (whose positionis controlled by a pointing device such as a mouse or touch pen) and amenu 51c are shown.

When the pointer 51b is shifted to an icon of "Set meshes" of the menu51c and then designated for selection, icons for inputting meshes aredisplayed on the picture, as shown by 51d in FIG. 51. When icons of"x-axis and y-axis" are selected by the pointer and further numericalvalues of intervals in X-axis and Y-axis directions are entered throughthe keyboard, meshes corresponding to the entered numerical values aredisplayed on the picture 51a. Further, it is also possible to store thenumbers of meshes previously as default values.

When the icon of "Set number of intermediate layer units" of the menu51c is selected by the pointer 51b in FIG. 51, a window 52d for settingthe number of intermediate layer units is displayed as shown in FIG. 52.Therefore, it is possible to set the number of the intermediate layerunits by shifting the pointer to this window and further entering anumerical value through the keyboard.

When the icon of "Set connection weight" of the menu 51c is selected bythe pointer 51b in FIG. 51, a window 53d for setting the connectionweight from the input layer unit to the intermediate layer unit isdisplayed as shown in FIG. 53. Further, a numerical value 53e indicativeof a number of the intermediate layer units is displayed in the window53d. Therefore, it is possible to change this numerical value byshifting the pointer to this window and further entering a numericalvalue through the keyboard. When the icon 53f of "Input" is selected andfurther a closed area is designated by the pointer in the picture 53a(on which an image of an object to be controlled is displayed), theconnection weights from the input layer units corresponding to theclosed area to the intermediate layer units are defined by randomnumbers whose averaged value is slightly larger than that in the otherareas. When an icon 53g is selected, the similar connection weights areset by random numbers whose averaged value is slightly smaller that inthe other areas. Here, the overlapped areas are handled as thenon-designated areas and set by random numbers. In FIG. 53, thesurrounding of the char bed shape required to be recognized is set to anarea designated by an icon 53f (+), and the other portions (to whichdust adhere) are set to areas designated by an icon 53g (-). Further,when the input icon 53h is selected, the setting of the connectionweights from the input layer units to one intermediate units ends.

When an icon 53i of "End menu" is selected, the coupling weight settingends. Here, when the number of the shaped to be recognized is smallerthan the number of intermediate layer units, as shown in FIG. 54, awarning message is displayed and further a window 54d for setting theconnection weights for the remaining intermediate layer units isdisplayed. In the window 54d, a number of the intermediate layer unitsand the number of setting image data pictures of the connection weightscorresponding thereto are displayed. Through selecting a number of theintermediate layer units by the pointer and further a numerical value isentered through the keyboard or by designating the bar graph by thepointer, it is possible to allocate the number of the pictures for theconnection weight setting so that the number of the picture correspondto the undefined intermediate layer units to the remaining intermediatelayer units. Accordingly,, it is possible to increase the recognitionpriority for the classification, by increasing the setting image datapictures corresponding to some classification. When the connectingweighs for the remaining intermediate layer units are not required to beset, an icon of "Random number" is selected. When an icon of "End" isselected in FIG. 54, the picture in FIG. 53 is shown again. Therefore,an icon of "End" is again selected in the menu 51c to end the setting.

After the connection weights from the input layer units to all theintermediate layer units have been set, a coupling weight displaypicture as shown in FIG. 55 is displayed. Here, if an icon of "Startlearning" (not shown) is selected, the learning of the neural networkstarts, and simultaneously the change of the connection weights aredisplayed in different colors corresponding to the magnitude of theconnection weights.

As described above, in the first embodiment of the one-loop controlleraccording to the present invention, it is possible to automate theoperation which is controlled by operator's manual decision now, andthereby to contribute to laborsaving.

In the second embodiment of the one-loop controller according to thepresent invention, since an interactive type interface is provided toadjust the control parameters such as membership functions and the rulesthrough the picture display interface, it is possible to reduce thelabor related to the control parameter adjustment, so that the adjustingwork of the actual system can be reduced.

In the third embodiment of the one-loop controller according to thepresent invention, since the interface pictures for executing the weightcoefficient learning function are provided for the neural network, it ispossible to reduce the labor required for the weight coefficientlearning in the neural network, so that the adjusting work of the actualsystem can be reduced.

In the fourth embodiment of the one-loop controller according to thepresent invention, since the control parameters can be adjusted by thesimulating function in the controller to some extent without operatingthe actual system, it is possible to reduce the load upon the controlleradjusting work markedly.

In the fifth embodiment of the one-loop controller according to thepresent invention, since the image display apparatus is accommodated onthe side surface of a rack and the display apparatus can be pulled outwhen necessary, it is possible to provide a large display picture easyto see without increasing the system space, so that the systemmanipulatability can be improved.

According to the image data display system according to the presentinvention, since a plurality of image data belonging to the samecategory can be displayed simultaneously so that the shapes thereof canbe compared with each other, it is possible to classify the image datamore effectively.

Further, in the connection weight input apparatus according to thepresent invention, since an object to be controlled is displayed on amonitor picture being divided into a plurality of areas and since aninterface picture is provided so that the connection weights can bedetermined by designating a group of the divided area, it is possible toset the initial connection weights between the nodes of the neuralnetworks more appropriately, thus improving the speed of the initiallearning process. In addition, since the function for setting the numberof image data pictures for initial value setting appropriately isprovided, it is possible to construct a fail-safe recognition system.

What is claimed is:
 1. A connection weight inputting apparatus,comprising:means for inputting, to a neural network for recognizing animage pattern, connection weights from a plurality of nodes of an inputlayer of the neural network to a plurality of nodes of intermediatelayers of the neural network; means for displaying an image to becontrolled on a display picture by dividing an image region into aplurality of regions according to the nodes of the input layer of theneural network; means for selecting at least one of the nodes of theintermediate layers of the neural network; means for determining theconnection weights for the intermediate layers containing the selectednodes by selecting a divided area on the display picture to input theconnection weights to the neural network.